A Maximum Certainty Approach to Feedforward NeuralNetworksStephen

A Bayesian-based methodology is presented which leads to a data analysis system based around a committee of radial-basis function (RBF) networks. We show that this approach enables estimatation of the uncertainty associated with system outputs. Systems with diiering numbers of internal degrees of freedom (weights) may hence be compared using training data only. Feedforward neural networks have become a widely recognised methodology for robust classiication, regression and prediction. This widespread use is accounted for in part by the fact that, if appropriate training data sets and error functions are used, the quantities which are estimated in response to a given input are close to optimal 1, 2, 3]. There are two drawbacks, however, to thètraditional' neural network approach. The rst being that the number of free parameters in the system (for feedforward networks the number of units in the hidden-layer) needs to be optimised in order to ensure that the network will generalise when presented with hitherto unseen inputs. The second drawback is the fact that, for many applications, the estimated output must be accompanied by some measure of error or certainty associated with the output. We present in this paper a simple system architecture, based upon a Bayesian learning methodology which leads naturally to a committee of networks. We show that on a typical problem this approach not only provides realistic error (uncertainty) estimates for the system's outputs but also a methodology whereby thèoptimal' number of hidden-layer units may be automatically assessed from a training set only.